Fast Algorithm for Estimating Mutual Information, Entropies and Score Functions

This papers proposes a fast algorithm for estimating the mutual information, difference score function, conditional score and conditional entropy, in possibly high dimensional space. The idea is to discretise the integral so that the density needs only be estimated over a regular grid, which can be done with little cost through the use of a cardinal spline kernel estimator. Score functions are then obtained as gradient of the entropy. An example of application to the blind separation of post-nonlinear mixture is given.